19 research outputs found
Light field super resolution through controlled micro-shifts of light field sensor
Light field cameras enable new capabilities, such as post-capture refocusing
and aperture control, through capturing directional and spatial distribution of
light rays in space. Micro-lens array based light field camera design is often
preferred due to its light transmission efficiency, cost-effectiveness and
compactness. One drawback of the micro-lens array based light field cameras is
low spatial resolution due to the fact that a single sensor is shared to
capture both spatial and angular information. To address the low spatial
resolution issue, we present a light field imaging approach, where multiple
light fields are captured and fused to improve the spatial resolution. For each
capture, the light field sensor is shifted by a pre-determined fraction of a
micro-lens size using an XY translation stage for optimal performance
Spatial and Angular Resolution Enhancement of Light Fields Using Convolutional Neural Networks
Light field imaging extends the traditional photography by capturing both
spatial and angular distribution of light, which enables new capabilities,
including post-capture refocusing, post-capture aperture control, and depth
estimation from a single shot. Micro-lens array (MLA) based light field cameras
offer a cost-effective approach to capture light field. A major drawback of MLA
based light field cameras is low spatial resolution, which is due to the fact
that a single image sensor is shared to capture both spatial and angular
information. In this paper, we present a learning based light field enhancement
approach. Both spatial and angular resolution of captured light field is
enhanced using convolutional neural networks. The proposed method is tested
with real light field data captured with a Lytro light field camera, clearly
demonstrating spatial and angular resolution improvement
HSTR-Net: Reference Based Video Super-resolution for Aerial Surveillance with Dual Cameras
Aerial surveillance requires high spatio-temporal resolution (HSTR) video for
more accurate detection and tracking of objects. This is especially true for
wide-area surveillance (WAS), where the surveyed region is large and the
objects of interest are small. This paper proposes a dual camera system for the
generation of HSTR video using reference-based super-resolution (RefSR). One
camera captures high spatial resolution low frame rate (HSLF) video while the
other captures low spatial resolution high frame rate (LSHF) video
simultaneously for the same scene. A novel deep learning architecture is
proposed to fuse HSLF and LSHF video feeds and synthesize HSTR video frames at
the output. The proposed model combines optical flow estimation and
(channel-wise and spatial) attention mechanisms to capture the fine motion and
intricate dependencies between frames of the two video feeds. Simulations show
that the proposed model provides significant improvement over existing
reference-based SR techniques in terms of PSNR and SSIM metrics. The method
also exhibits sufficient frames per second (FPS) for WAS when deployed on a
power-constrained drone equipped with dual cameras.Comment: 15 pages, 8 figures, 8 table
Multi-frame information fusion for image and video enhancement
Ph.D.Committee Chair: Yucel Altunbasa
Superresolution under Photometric Diversity of Images
Superresolution (SR) is a well-known technique to increase the quality of an image using multiple overlapping pictures of a scene. SR requires accurate registration of the images, both geometrically and photometrically. Most of the SR articles in the literature have considered geometric registration only, assuming that images are captured under the same photometric conditions. This is not necessarily true as external illumination conditions and/or camera parameters (such as exposure time, aperture size, and white balancing) may vary for different input images. Therefore, photometric modeling is a necessary task for superresolution. In this paper, we investigate superresolution image reconstruction when there is photometric variation among input images